Oblivious Bounds on the Probability of Boolean Functions

This paper develops upper and lower bounds for the probability of Boolean functions by treating multiple occurrences of variables as independent and assigning them new individual probabilities. We call this approach dissociation and give an exact characterization of optimal oblivious bounds, i.e. when the new probabilities are chosen independent of the probabilities of all other variables. Our motivation comes from the weighted model counting problem (or, equivalently, the problem of computing the probability of a Boolean function), which is #P-hard in general. By performing several dissociations, one can transform a Boolean formula whose probability is difficult to compute, into one whose probability is easy to compute, and which is guaranteed to provide an upper or lower bound on the probability of the original formula by choosing appropriate probabilities for the dissociated variables. Our new bounds shed light on the connection between previous relaxation-based and model-based approximations and unify them as concrete choices in a larger design space. We also show how our theory allows a standard relational database management system (DBMS) to both upper and lower bound hard probabilistic queries in guaranteed polynomial time.Comment: 34 pages, 14 figures, supersedes: http://arxiv.org/abs/1105.281

Hinge-Loss Markov Random Fields and Probabilistic Soft Logic: A Scalable Approach to Structured Prediction

A fundamental challenge in developing impactful artificial intelligence technologies is balancing the ability to model rich, structured domains with the ability to scale to big data. Many important problem areas are both richly structured and large scale, from social and biological networks, to knowledge graphs and the Web, to images, video, and natural language. In this thesis I introduce two new formalisms for modeling structured data, distinguished from previous approaches by their ability to both capture rich structure and scale to big data. The first, hinge-loss Markov random fields (HL-MRFs), is a new kind of probabilistic graphical model that generalizes different approaches to convex inference. I unite three views of inference from the randomized algorithms, probabilistic graphical models, and fuzzy logic communities, showing that all three views lead to the same inference objective. I then derive HL-MRFs by generalizing this unified objective. The second new formalism, probabilistic soft logic (PSL), is a probabilistic programming language that makes HL-MRFs easy to define, refine, and reuse for relational data. PSL uses a syntax based on first-order logic to compactly specify complex models. I next introduce an algorithm for inferring most-probable variable assignments (MAP inference) for HL-MRFs that is extremely scalable, much more so than commercially available software, because it uses message passing to leverage the sparse dependency structures common in inference tasks. I then show how to learn the parameters of HL-MRFs using a number of learning objectives. The learned HL-MRFs are as accurate as traditional, discrete models, but much more scalable. To enable HL-MRFs and PSL to capture even richer dependencies, I then extend learning to support latent variables, i.e., variables without training labels. To overcome the bottleneck of repeated inferences required during learning, I introduce paired-dual learning, which interleaves inference and parameter updates. Paired-dual learning learns accurate models and is also scalable, often completing before traditional methods make even one parameter update. Together, these algorithms enable HL-MRFs and PSL to model rich, structured data at scales not previously possible

Construction of Energy Functions for Lattice Heteropolymer Models: Efficient Encodings for Constraint Satisfaction Programming and Quantum Annealing

Optimization problems associated with the interaction of linked particles are at the heart of polymer science, protein folding and other important problems in the physical sciences. In this review we explain how to recast these problems as constraint satisfaction problems such as linear programming, maximum satisfiability, and pseudo-boolean optimization. By encoding problems this way, one can leverage substantial insight and powerful solvers from the computer science community which studies constraint programming for diverse applications such as logistics, scheduling, artificial intelligence, and circuit design. We demonstrate how to constrain and embed lattice heteropolymer problems using several strategies. Each strikes a unique balance between number of constraints, complexity of constraints, and number of variables. In addition, each strategy has distinct advantages and disadvantages depending on problem size and available resources. Finally, we show how to reduce the locality of couplings in these energy functions so they can be realized as Hamiltonians on existing adiabatic quantum annealing machines.Chemistry and Chemical Biolog

Community detection and stochastic block models: recent developments

The stochastic block model (SBM) is a random graph model with planted clusters. It is widely employed as a canonical model to study clustering and community detection, and provides generally a fertile ground to study the statistical and computational tradeoffs that arise in network and data sciences. This note surveys the recent developments that establish the fundamental limits for community detection in the SBM, both with respect to information-theoretic and computational thresholds, and for various recovery requirements such as exact, partial and weak recovery (a.k.a., detection). The main results discussed are the phase transitions for exact recovery at the Chernoff-Hellinger threshold, the phase transition for weak recovery at the Kesten-Stigum threshold, the optimal distortion-SNR tradeoff for partial recovery, the learning of the SBM parameters and the gap between information-theoretic and computational thresholds. The note also covers some of the algorithms developed in the quest of achieving the limits, in particular two-round algorithms via graph-splitting, semi-definite programming, linearized belief propagation, classical and nonbacktracking spectral methods. A few open problems are also discussed

Proceedings of the 18th Irish Conference on Artificial Intelligence and Cognitive Science

These proceedings contain the papers that were accepted for publication at AICS-2007, the 18th Annual Conference on Artificial Intelligence and Cognitive Science, which was held in the Technological University Dublin; Dublin, Ireland; on the 29th to the 31st August 2007. AICS is the annual conference of the Artificial Intelligence Association of Ireland (AIAI)

LIPIcs, Volume 261, ICALP 2023, Complete Volume

LIPIcs, Volume 261, ICALP 2023, Complete Volum

Efficient techniques for statistical modeling of calibration and spatio-temporal systems using Gaussian processes

Gaussian processes (GPs) are one of the most widely used tools in statistical modeling of various engineering systems. In this dissertation, we study three common types of problems in statistical modeling, i.e., prediction, calibration, and forecasting, using GPs and other related techniques.First, we study the problem of prediction using Gaussian Process Regression (GPR) in large-scale spatial systems that contain exogenous variables. We propose a Sparse Pseudo-input Local Gaussian Process (SPLGP) that addresses the inefficiencies of GPR, i.e., computational complexity and covariance heterogeneity, in dealing with spatial systems in a unifying framework. We propose new theorems that form the basis of our decomposition policy and develop an optimization procedure to find the optimal policy. We also impose continuity constraints on the boundaries of the subdomains to alleviate the problem of discontinuity of the global predictor. Next, we study the calibration problem for expensive computational models (ECM), i.e., computational models that cannot be evaluated a large number of times. We propose a Bayesian Non-isometric Matching Calibration (BNMC) approach that allows calibration of ECM. The proposed model uses GPs to embrace the restrictions of ECM and makes inferences on the calibration parameters through a Bayesian framework. We also present a geometric interpretation of calibration that enables us to take advantage of combinatorial optimization techniques to extract necessary information for constructing prior distributions of our Bayesian framework. Finally, we study the problem of forecasting in complex spatio-temporal systems with the primary focus on short-term wind speed forecasting in wind farms. We propose a similarity-based forecasting model capable of taking any type of spatial and temporal information into account to improve spatio-temporal forecasting, in particular wind speed forecasting. The proposed model is inspired by the weighted averaging technique used in a class of regression models known as non-parametric linear smoothers which includes GPR. We also equip our model with a variable selection and a parameter training procedure, so that it can be easily applied to any spatio-temporal system. We present a set of experimental results for each problem to demonstrate the efficiency of our proposed models comparing to other existing models