Monte Carlo Co-Ordinate Ascent Variational Inference

In Variational Inference (VI), coordinate-ascent and gradient-based approaches are two major types of algorithms for approximating difficult-to-compute probability densities. In real-world implementations of complex models, Monte Carlo methods are widely used to estimate expectations in coordinate-ascent approaches and gradients in derivative-driven ones. We discuss a Monte Carlo Co-ordinate Ascent VI (MC-CAVI) algorithm that makes use of Markov chain Monte Carlo (MCMC) methods in the calculation of expectations required within Co-ordinate Ascent VI (CAVI). We show that, under regularity conditions, an MC-CAVI recursion will get arbitrarily close to a maximiser of the evidence lower bound (ELBO) with any given high probability. In numerical examples, the performance of MC-CAVI algorithm is compared with that of MCMC and -- as a representative of derivative-based VI methods -- of Black Box VI (BBVI). We discuss and demonstrate MC-CAVI's suitability for models with hard constraints in simulated and real examples. We compare MC-CAVI's performance with that of MCMC in an important complex model used in Nuclear Magnetic Resonance (NMR) spectroscopy data analysis -- BBVI is nearly impossible to be employed in this setting due to the hard constraints involved in the model

Bayesian Inference for Duplication-Mutation with Complementarity Network Models

We observe an undirected graph $G$ without multiple edges and self-loops, which is to represent a protein-protein interaction (PPI) network. We assume that $G$ evolved under the duplication-mutation with complementarity (DMC) model from a seed graph, $G_0$, and we also observe the binary forest $\Gamma$ that represents the duplication history of $G$. A posterior density for the DMC model parameters is established, and we outline a sampling strategy by which one can perform Bayesian inference; that sampling strategy employs a particle marginal Metropolis-Hastings (PMMH) algorithm. We test our methodology on numerical examples to demonstrate a high accuracy and precision in the inference of the DMC model's mutation and homodimerization parameters

A new damage index for plane steel frames exhibiting strength and stiffness degradation under seismic motion

A new damage index for plane steel frames under earthquake ground motion is proposed. This index is defined at a section of a steel member and takes into account the interaction between the axial force N and bending moment M acting there. This interaction is defined by two curves in the N-. M plane. The first curve is the limit between elastic and inelastic material behavior, where damage is zero, while the second one is the limit between inelastic behavior and complete failure, where damage is equal to one. The damage index is defined by assuming a linear variation of damage between the two abovementioned curves. Thus, for a given N-. M combination at a member section, obtained with the aid of a two dimensional finite element program, one easily defines the damage index of that section. Material nonlinearities are taken into account by a stress-strain bilinear model including cyclic strength and stiffness degradation in the framework of lumped plasticity (plastic hinge model), while geometrical nonlinearities are modeled by including large deflection effects. The increase of damage related to strength reduction due to low-cycle fatigue is also taken into account. Several illustrative examples serve to demonstrate the use of the proposed damage index and to compare it with other well known damage indices. © 2012

Bayesian Learning of Graph Substructures

Graphical models provide a powerful methodology for learning the conditional independence structure in multivariate data. Inference is often focused on estimating individual edges in the latent graph. Nonetheless, there is increasing interest in inferring more complex structures, such as communities, for multiple reasons, including more effective information retrieval and better interpretability. Stochastic blockmodels offer a powerful tool to detect such structure in a network. We thus propose to exploit advances in random graph theory and embed them within the graphical models framework. A consequence of this approach is the propagation of the uncertainty in graph estimation to large-scale structure learning. We consider Bayesian nonparametric stochastic blockmodels as priors on the graph. We extend such models to consider clique-based blocks and to multiple graph settings introducing a novel prior process based on a dependent Dirichlet process. Moreover, we devise a tailored computation strategy of Bayes factors for block structure based on the Savage-Dickey ratio to test for presence of larger structure in a graph. We demonstrate our approach in simulations as well as on real data applications in finance and transcriptomics.Comment: 35 pages, 7 figure

Graph Sphere: From Nodes to Supernodes in Graphical Models

High-dimensional data analysis typically focuses on low-dimensional structure, often to aid interpretation and computational efficiency. Graphical models provide a powerful methodology for learning the conditional independence structure in multivariate data by representing variables as nodes and dependencies as edges. Inference is often focused on individual edges in the latent graph. Nonetheless, there is increasing interest in determining more complex structures, such as communities of nodes, for multiple reasons, including more effective information retrieval and better interpretability. In this work, we propose a multilayer graphical model where we first cluster nodes and then, at the second layer, investigate the relationships among groups of nodes. Specifically, nodes are partitioned into "supernodes" with a data-coherent size-biased tessellation prior which combines ideas from Bayesian nonparametrics and Voronoi tessellations. This construct allows accounting also for dependence of nodes within supernodes. At the second layer, dependence structure among supernodes is modelled through a Gaussian graphical model, where the focus of inference is on "superedges". We provide theoretical justification for our modelling choices. We design tailored Markov chain Monte Carlo schemes, which also enable parallel computations. We demonstrate the effectiveness of our approach for large-scale structure learning in simulations and a transcriptomics application.Comment: 71 pages, 18 figure

Bridging trees for posterior inference on Ancestral Recombination Graphs

We present a new Markov chain Monte Carlo algorithm, implemented in software Arbores, for inferring the history of a sample of DNA sequences. Our principal innovation is a bridging procedure, previously applied only for simple stochastic processes, in which the local computations within a bridge can proceed independently of the rest of the DNA sequence, facilitating large-scale parallelisation.Comment: 23 pages, 9 figures, accepted for publication in Proceedings of the Royal Society

Change point detection in dynamic Gaussian graphical models: the impact of COVID-19 pandemic on the US stock market

Reliable estimates of volatility and correlation are fundamental in economics and finance for understanding the impact of macroeconomics events on the market and guiding future investments and policies. Dependence across financial returns is likely to be subject to sudden structural changes, especially in correspondence with major global events, such as the COVID-19 pandemic. In this work, we are interested in capturing abrupt changes over time in the dependence across US industry stock portfolios, over a time horizon that covers the COVID-19 pandemic. The selected stocks give a comprehensive picture of the US stock market. To this end, we develop a Bayesian multivariate stochastic volatility model based on a time-varying sequence of graphs capturing the evolution of the dependence structure. The model builds on the Gaussian graphical models and the random change points literature. In particular, we treat the number, the position of change points, and the graphs as object of posterior inference, allowing for sparsity in graph recovery and change point detection. The high dimension of the parameter space poses complex computational challenges. However, the model admits a hidden Markov model formulation. This leads to the development of an efficient computational strategy, based on a combination of sequential Monte-Carlo and Markov chain Monte-Carlo techniques. Model and computational development are widely applicable, beyond the scope of the application of interest in this work

Direct damage-controlled design of plane steel moment-resisting frames using static inelastic analysis

A new direct damage-controlled design method for plane steel frames under static loading is presented. Seismic loading can be handled statically in the framework of a push-over analysis. This method, in contrast to existing steel design methods, is capable of directly controlling damage, both local and global, by incorporating continuum damage mechanics for ductile materials in the analysis. The design process is accomplished with the aid of a two-dimensional finite element program, which takes into account material and geometric nonlinearities by using a nonlinear stress-strain relation through the beam-column fiber modeling and including P-δ and P-Δ effects, respectively. Simple expressions relating damage to the plastic hinge rotation of member sections and the interstorey drift ratio for three performance limit states are derived by conducting extensive parametric studies involving plane steel moment-resisting frames under static loading. Thus, a quantitative damage scale for design purposes is established. Using the proposed design method one can either determine damage for a given structure and loading, or dimension a structure for a target damage and given loading, or determine the maximum loading for a given structure and a target damage level. Several numerical examples serve to illustrate the proposed design method and demonstrate its advantages in practical applications

Geodesic Monte Carlo on embedded manifolds

Markov chain Monte Carlo methods explicitly defined on the manifold of probability distributions have recently been established. These methods are constructed from diffusions across the manifold and the solution of the equations describing geodesic flows in the Hamilton–Jacobi representation. This paper takes the differential geometric basis of Markov chain Monte Carlo further by considering methods to simulate from probability distributions that themselves are defined on a manifold, with common examples being classes of distributions describing directional statistics. Proposal mechanisms are developed based on the geodesic flows over the manifolds of support for the distributions, and illustrative examples are provided for the hypersphere and Stiefel manifold of orthonormal matrices